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IN THE CLAIMS 

1 . (Currently Amended) A comput e r impl e m e nt e d method of reducing the number 
of hypotheses for locations of an active object in a saved image generated by a graphical user 
interface (GUI) of an application program comprising: 

determining possible triangles for the saved image, wherein vertices of the triangles are 
points where the active object and any two additional objects of the saved image are located; 

determining possible pairs of hypotheses corresponding to the two additional objects 
from a current playback image corresponding to the saved image; 

determining triangles corresponding to the possible pairs of hypotheses and filtering the 
detected triangles; 

determining vertices with weights for similar triangles of the current playback image; and 
calculatin g, at a processor, a weight for every hypothesis of the active object and filtering 
hypotheses of the active object on the current playback image based on a weight bound. 

2. (Original) The method of claim 1 5 wherein determining possible triangles 
comprises decreasing the number of possible triangles by using only triangles having an edge 
between two additional objects whose length is less than a predetermined distance. 

3. (Original) The method of claim 1, wherein determining possible pairs of 
hypotheses comprises decreasing the number of possible triangles by using only triangles which 
have an edge between two additional objects whose length is within a calculated interval based 
on a first coefficient value. 

4. (Original) The method of claim 1, wherein determining triangles corresponding to 
the pairs of hypotheses comprises decreasing the number of triangles by using only triangles 
which have angles within a calculated interval based on a second coefficient value. 

5. (Original) The method of claim 1, wherein a weight of a selected vertex of a 
triangle comprises 1 .0 multiplied by the number of vertices of other triangles coincident at the 
selected vertex. 

6. (Original) The method of claim 1 , wherein calculating a weight for every 
hypothesis of the active object comprises calculating the sum of distances between the location 
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of the hypothesis and all corresponding vertices using a third coefficient value and an upper 
bound value. 

7. (Original) The method of claim 1, wherein filtering the hypotheses of the active 
object comprises filtering a hypothesis if the calculated weight of the hypothesis is less than the 
weight bound. 

8. (Original) The method of claim 1, further comprising identifying the best 
hypothesis for the active object based on the filtering. 

9. (Currently Amended) An article comprising: a storage medium readable by a 
processor comprising machin e acc e ssibl e m e dium containing instructions, which when executed, 
result in reducing the number of hypotheses for locations of an active object in a saved image 
generated by a graphical user interface (GUI) of an application program by 

determining possible triangles for the saved image, wherein vertices of the triangles are 
points where the active object and any two additional objects of the saved image are located; 

determining possible pairs of hypotheses corresponding to the two additional objects 
from a current playback image corresponding to the saved image; 

determining triangles corresponding to the possible pairs of hypotheses and filtering the 
detected triangles; 

determining vertices with weights for similar triangles of the current playback image; and 
calculating a weight for every hypothesis of the active object and filtering 
hypotheses of the active object on the current playback image based on a weight bound. 

10. (Original) The article of claim 9, wherein instructions to determine possible 
triangles comprise instructions to decrease the number of possible triangles by using only 
triangles having an edge between two additional objects whose length is less than a 
predetermined distance. 

1 1 . (Original) The article of claim 9, wherein instructions to determine possible pairs 
of hypotheses comprise instructions to decrease the number of possible triangles by using only 
triangles which have an edge between two additional objects whose length is within a calculated 
interval based on a first coefficient value. 

12. (Original) The article of claim 9, wherein instructions to determine triangles 
corresponding to the pairs of hypotheses comprise instructions to decrease the number of 
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triangles by using only triangles which have angles within a calculated interval based on a 
second coefficient value. 

13. (Original) The article of claim 9, wherein a weight of a selected vertex of a 
triangle comprises 1 .0 multiplied by the number of vertices of other triangles coincident at the 
selected vertex. 

14. (Original) The article of claim 9, wherein instructions to calculate a weight for 
every hypothesis of the active object comprise instructions to calculate the sum of distances 
between the location of the hypothesis and all corresponding vertices using a third coefficient 
value and an upper bound value. 

15. (Original) The article of claim 9, wherein instructions to filter the hypotheses of 
the active object comprise instructions to filter a hypothesis if the calculated weight of the 
hypothesis is less than the weight bound. 

16. (Original) The article of claim 9, further comprising instructions to identify the 
best hypothesis for the active object based on the filtering. 

1 7. (Currently Amended) A cognitive control framework system for automatically 
controlling execution of an application program having a graphical user interface comprising: 

a recording component adapted to capture user input data and images displayed by the 
graphical user interface during a recording phase of execution of the application program, and to 
analyze the captured user input data and displayed images to generate an execution scenario 
during the recording phase; and 

a playback component adapted to perform image analysis on images displayed by the 
graphical user interface as a result of processing the simulated user input data during the 
playback phase and captured displayed images from the recording phase by, the playback 
component being adapted to reduce the number of hypotheses for locations of an active object in 
a saved image generated by a graphical user interface (GUI) of an application program by , at a 
processor: 

determining possible triangles for the saved image, wherein vertices of the 
triangles are points where the active object and any two additional objects of the saved 
image are located; 
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determining possible pairs of hypotheses corresponding to the two additional 
objects from a current playback image corresponding to the saved image; 

determining triangles corresponding to the possible pairs of hypotheses and 
filtering the detected triangles; 

determining vertices with weights for similar triangles of the current playback 
image; and 

calculating a weight for every hypothesis of the active object and filtering 
hypotheses of the active object on the current playback image based on a weight bound. 

18. (Original) The system of claim 17, wherein determining possible triangles 
comprises decreasing the number of possible triangles by using only triangles having an edge 
between two additional objects whose length is less than a predetermined distance. 

19. (Original) The system of claim 17, wherein determining possible pairs of 
hypotheses comprises decreasing the number of possible triangles by using only triangles which 
have an edge between two additional objects whose length is within a calculated interval based 
on a first coefficient value. 

20. (Original) The system of claim 17, wherein determining triangles corresponding 
to the pairs of hypotheses comprises decreasing the number of triangles by using only triangles 
which have angles within a calculated interval based on a second coefficient value. 

21. (Original) The system of claim 17, wherein a weight of a selected vertex of a 
triangle comprises 1 .0 multiplied by the number of vertices of other triangles coincident at the 
selected vertex. 

22. (Original) The system of claim 17, wherein calculating a weight for every 
hypothesis of the active object comprises calculating the sum of distances between the location 
of the hypothesis and all corresponding vertices using a third coefficient value and an upper 
bound value. 

23. (Original) The system of claim 17, wherein filtering the hypotheses of the active 
object comprises filtering a hypothesis if the calculated weight of the hypothesis is less than the 
weight bound. 

24. (Original) The system of claim 1 7, wherein the playback component is adapted to 
identify the best hypothesis for the active object based on the filtering. 



